Clustering dialects automatically: A mutual information approach

Dialects can be categorized in many ways. Using external features, dialects may be grouped by geographic location (e.g., Irish English), ethnic identity (e.g., AAVE), or social networks (e.g., Liberian Settler English) of their speakers. Or, using internal features, dialects may be grouped by shared features of pronunciation, vocabulary, or grammar. We explore quantitative approaches to see how similarly dialects cluster by these different methods. We describe a method of clustering dialects according to patterns of shared phonological features. While previous linguistic research has generally considered such phonological features as independent of each other, we examine their statistical co-variation. That is, we look at the degree to which variation in one feature predicts variation in each other feature, or Mutual Information (MI). As an example, we look at the degree to which we can predict whether a dialect will exhibit the cot/caught merger based on knowledge of whether they vocalize /r/ in the word barn. Within phonological theory, these variables are independent of each other, but they do exhibit statistical dependence. To test our method, we explore a data set consisting of 168 binary features describing the pronunciation of vowels and consonants of English speakers from 35 countries and regions. This is a subset of the data collected for the Handbook of Varieties of English (Schneider et al. 2005). These dialects are grouped according to patterns of shared features. The results of this method of categorizing dialect varieties by binary pronunciation features are compared to traditional groupings based on external features. In many ways, the clusters produced by this method are similar. We also compare differences in clustering outcomes determined by phonological vs. morphosyntactic features, as well as differences that depend on the method of clustering.1